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Abstract Body 


Background / Context: 

These studies are especially well-suited for the spring 2013 SREE conference theme - 
Capitalizing on Contradiction: Eearning from Mixed Results. In one, the first, large randomized 
controlled trial in higher education, MDRC tested the impact of learning communities at 
Kingsborough Community College in New York. In part because of early positive findings at 
Kingsborough, the National Center for Postsecondary Research, of which MDRC is a partner, 
launched additional tests of learning community programs in six colleges. In contrast to the 
early results at Kingsborough, short-term findings in these trials were, on average, much more 
modest. Eollow-up of the sample at Kingsborough, however, demonstrated positive long-term 
impacts: after six years, the estimated impact on degree completion was 4.6 percentage points. 
The present research explores similarities and differences across these six randomized controlled 
trials of learning communities in community colleges. It examines several competing 
explanations for the mixed findings and draws lessons to inform further research and the field. 

Over the last 50 years, community colleges have played an increasingly vital role in American 
postsecondary education. Each fall, community colleges now enroll 35 percent of all 
postsecondary education students (Provasnik & Planty, 2008). Unfortunately, while enrollments 
are increasing, overall success rates in community colleges are disappointingly low. Among first- 
time students who enrolled in community colleges during the 2003-2004 academic year, only 
about a third earned a degree or certificate in six years (U.S. Department of Education, 201 1). 
While the rates of degree or certificate attainment are low in general, rates are even lower for 
students in need of developmental education, who comprise a significant proportion of the 
community college student body (Adelman, 2004; Attewell, et ah, 2006). 

Eearning communities, which consist of groups of students who enroll together in two or more 
linked classes, are a popular reform aimed to improve students’ academic outcomes. These 
linked classes often incorporate shared assignments and curricula, collaboration between faculty 
teaching pairs, and connections to student support services. The theory of change for learning 
communities predicts that students in them will become more engaged in what they are learning 
and more connected with each other and with their instructors. The theory predicts that students 
will then be more likely to develop higher-order thinking skills, master the course material, pass 
their classes, and persist from semester to semester (Visher et ah, 2008; Smith et ah, 2004). 

Prior research on learning communities in community colleges includes theoretical work, 
nonexperimental studies, and a few quasi-experimental studies of the effects of learning 
communities on psychological and academic outcomes, but no randomized trials. The reported 
benefits of learning communities include more positive feelings about the college experience, 
increased levels of engagement, more meaningful relationships among students, increased 
interaction around academic activity, stronger recognition of connections across disciplines, and 
improved higher-order thinking skills (Visher et al., 2008; Smith et ah, 2004). Academic 
outcomes have also been evaluated in nonexperimental and quasiexperimental studies. These 
studies suggest that learning communities may have modest positive associations with outcomes 
such as course completion, grades, and persistence in college (Gordon, Young, and Kalianov, 
2001; Tinto, 1997; Tinto, 1998; Engstrom and Tinto, 2008). While these findings are promising. 
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the designs of these studies leave open the question of whether their effects are due to the 
program itself or to preprogram differences in the characteristics of those students who chose to 
enroll in the program (such as their academic ability or motivation levels). 

Purpose / Objective / Research Question / Focus of Study: 

This research adds to the knowledge base, offering reliable evidence from six randomized 
controlled studies at different colleges implementing learning communities, most of which 
included developmental education classes (classes for students found to be underprepared for 
college-level material). Five of these studies track students’ progress in the program semester 
and two follow-up semesters, and one study follows students for six years. These studies 
together provide the most extensive evidence available on the promise and limitations of learning 
communities for improving the academic outcomes of students in community colleges. 

Setting: 

The research took place in six community colleges across the United States. 

• The Community College of Baltimore County (CCBC) comprises three main campuses 
and three additional locations in suburban Maryland, near the city of Baltimore. It serves 
about 20,000 students each fall. 

• Hillsborough Community College is an urban community college located in Tampa, 
Florida. It serves approximately 20,000 students each fall. 

• Houston Community College, the largest college in the study, is a community college 
system comprised of six colleges located in and around Houston, Texas. These colleges, 
several of which have multiple campuses, serve over 40,000 students each fall. 

• Kingsborough Community College, located in Brooklyn, New York, is one of seven 
community colleges in the City University of New York (CUNY) system. 

Kingsborough’ s student enrollment each fall is approximately 15,000. 

• Merced College is a midsized college in Merced, in California’s agricultural Central 
Valley. It serves about 1 1,000 students each fall, and is the smallest college in this study. 

• Queensborough Community College, a midsized college in Queens, New York, serves 
over 13,000 students each fall and, like Kingsborough, is part of the CUNY system. 

Population / Participants / Subjects: 

The trials in these studies constitute one of the largest experiments of an intervention in higher 
education: more than 7,000 students are included across the six sites, nearly all of whom required 
some developmental education classes. Study participants came from a diverse set of 
backgrounds and experiences. Study participants were primarily of traditional college age at 
baseline and were racially diverse. Many study participants were the first person in their family 
to attend college. For more details on the study participants, see Tables 1 and 2. 

Intervention / Program / Practice: 

The learning communities each lasted for one semester, and consisted of four key components, 
although there was variation in their emphasis and implementation: 
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• Linked Courses and Student Cohorts; The learning communities consisted of groups 
of students who enrolled together in two or more courses, often scheduled back-to-back. 

• Faculty Collaboration: As part of a learning community, instructors of the linked 
courses collaborated to plan and run their classes. 

• Instructional Practices: Teaching methods in the learning communities focused on 
integrated instruction, emphasizing the interconnections between the linked courses, and 
active and collaborative learning, including strategies like group work and hands-on 
assignments. Instructors tried to foster integrated learning by connecting course content 
with the other course in the link and with students’ own lives and the world around them. 

• Student Supports: Some learning communities programs included enhanced student 
support services that provide students with increased access to tutors, designated 
counselors, or supplemental instruction in the classroom. 

Research Design: 

Students were assigned, at random, either to the program group, whose members had the 
opportunity to participate in learning communities, or to the control group, whose members 
received the college’s standard services. Each college’s research sample was comprised of three 
or four groups (or cohorts) of students. Each cohort started at the beginning of subsequent 
semesters (e.g. fall 2007, spring 2008, etc.). The random assignment process occurred separately 
at each college for each cohort and was controlled by MDRC. The Kingsborough students were 
tracked for six years. Students at the other colleges were tracked for three semesters.^ 

Data Collection and Analysis: 

The research uses several data sources: 

• The Baseline Information Form: Just before random assignment, students completed a 
short survey to collect information on demographic and other background characteristics. 

• Operational Site Visits, Field Research, and Instructor Survey: The research team 
interviewed college administrators, faculty, and staff and students involved in the 
learning communities, and conducted informal observations of some learning community 
classes. A survey of instructors in learning communities was also administered. 

• Student Records: Each college provided information on program and control group 
students’ academic outcomes from their student transcripts. The six-year follow-up at 
Kingsborough includes CUNY transcript data and National Student Clearinghouse data. 

Findings / Results: 

• Long-Term Findings from Kingsborough Community College: After six years, more 
students in the learning communities earned a degree (35.9 percent) than did students in 
the control group (31.3 percent) — an impact of 4.6 percentage points. The increase in 


* At Kingsborough, some learning communities did not include developmental education classes, while at the other 
colleges all learning communities included developmental education classes. The analyses of short-term outcomes 
include these outcomes for the sample of students at Kingsborough who placed into developmental English. 
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degree receipt was most evident for students who placed into college-level English at 
baseline, although there is evidence that the program also improved the long-term 
outcomes of students with the greatest developmental needs in English. The program also 
increased average credit accumulation by 4.0 credits over six years (see Tables 3 and 4). 

• Short-Term Findings from Six Community Colleges That Operated Learning 
Communities with Developmental English or Math: On average, these programs 
produced no impact on persistence, a half-credit impact on credits earned in the targeted 
subject (English or mathematics), no impact on credits outside that subject, and a half- 
credit effect on total credits earned. There was also evidence that program effects varied 
by college with respect to credits earned in the targeted subject (with the estimated 
impacts at Kingsborough being the largest), but no evidence of discernible variation in 
impacts on the other key short term outcomes (see Eigures 1-4). 

Conclusions: 

Taken together, the results from the studies suggest the following: 

• One-semester learning communities can have a long-term impact and even boost 
graduation, as shown in the study of the Kingsborough program. 

• However, the combined results all six trials suggest that on average learning communities 
for developmental education students produce only a modest impact on credits earned in 
the targeted subject of English or mathematics. 

Notably, these studies purposely selected programs that represent a range of typical learning 
community programs as they exist in community colleges. As a result, the research provides 
good tests of learning communities as they appear to be typically enacted, but not a test of the 
“ideal” or “advanced” learning communities described in the literature. 

Together these trials produce highly reliable estimates of the impacts of six programs, both 
individually and as a whole. However, important questions remain. In particular, was there 
something about the Kingsborough program or its setting that accounts for the larger impacts in 
the short term and the impact on completion? The program at Kingsborough Community College 
was the subject of the long-term follow-up, and also the program with the largest estimated 
effects in the short term. While a number of factors may have played a role in the impacts at 
Kingsborough, the studies cannot pinpoint which of these features mattered most. 

This research explores a number of competing explanations to help understand the mixed results 
across the six randomized trials. Together, analyses of these studies suggest important lessons, 
both for researchers confronted with mixed results from multiple trials, and for community 
college practitioners who are considering implementing learning communities for their students. 
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Appendix B. Tables and Figures 


Table 1: Baseline Characteristics of Students in Sample, by College 



CCBC Hillsborough Houston 

Kmgsborough Merced 

Queensboiough 

Gender (f a) 







Male 

41.3 

43.1 

33.3 

48.6 

48.7 

44.1 

Female 

58.7 

56.9 

66.7 

51.4 

51.3 

55.9 

Age (%) 







20 years old and younger 

77.1 

70.2 

63.0 

75.5 

65.4 

78.1 

21-25 \'ears old 

12.7 

16.2 

18.4 

18.2 

17.1 

15.2 

26 - 30 \ears old 

4.3 

5.6 

8.7 

3.6 

6.3 

3.2 

3 1 and older 

5.9 

8.0 

9.9 

2.8 

11.3 

3.6 

Average age (years) 

22.0 

20.4 

22.2 

21.2 

19.9 

20.0 

Race, ethmcitv* (*/•) 







Hispanic 

44 

31.8 

54.8 

19.8 

54.9 

32.8 

White 

31.5 

24.5 

3.1 

22.6 

16.5 

13.6 

Black 

54.9 

36.6 

34.4 

38.3 

89 

30.7 

Asian or Pacific Islander 

2.1 

3.7 

0.8 

9.1 

12.9 

11.9 

Other” 

3.8 

2.3 

0.8 

6.0 

3.2 

5.1 

Missing 

3.3 

1.0 

6.2 

4.2 

3.6 

5.9 

Single parent (••) 

119 

13.9 

21.2 

7.7 

18.3 

3.8 

Missing 

17.5 

15.6 

193 

3.9 

18.8 

25.2 

Has one or more children (* o) 

15.1 

18.7 

28.2 

9.4 

26.2 

7.4 

Missing 

2.8 

2.3 

6.5 

11 

4.6 

2.7 

Average age of youngest child (years) 

4.3 

5.3 

5.2 

4.6 

2.9 

3.5 

Missing 

0.0 

0.0 

4.3 

2.5 

2.9 

5.3 

Received financial aid during semester of random assignment (%) 

449 

25.0 

43.1 

N.'A' 

26.8 

27.2 

Missing 

22.4 

34.1 

29.9 

N/A' 

30.2 

37.9 

Financially dependent on pat ents (%) 

41 1 

35.0 

29.1 

72.3 

31.8 

37.0 

Missing 

148 

16.4 

18.0 

1.0 

20.2 

29.6 

Highest grade conqileted (%) 







1 1th grade or lower 

5.9 

12.0 

12.0 

27.1 

7.9 

15 1 

12 th grade 

90.9 

85.4 

80.8 

71.4 

87.0 

78.4 

Missing 

3.2 

2.5 

7.2 

1.5 

5 1 

6.5 

Diplomas degrees earned'* (“/b) 







GED 

1.9 

13.7 

11.8 

31 1 

7.3 

168 

High school diploma 

87.9 

82.2 

78.2 

67.5 

78.6 

75.7 

Occupational technical certificate 

5.1 

6.4 

5.6 

2.1 

4.0 

2.7 

Two-year or higher degree 

0.1 

1.0 

0.7 

0.1 

0.6 

0.1 

None of the above 

13 

1.1 

2.5 

0.3 

5.0 

2.4 

Missmg 

3.1 

2.5 

64 

0.0 

5.8 

4.0 

Taken any college courses (5 b) 

23.5 

8.7 

12.1 

5.0 

35.0 

21 4 

Missing 

3.0 

1.9 

6.7 

1.3 

4.9 

4.6 

First person in family to attend college (5b) 

26.2 

29.7 

40.3 

34.3 

35.6 

24.6 

Missing 

4.0 

4.1 

8.3 

3.1 

5.9 

7.7 

Working personal computer m home (5b) 

84.8 

83.9 

64.8 

698 

63.9 

84.8 

Missing 

2.9 

2.2 

6.5 

9.9 

44 

3.7 

Language other than Enghsh spoken regularly m home (5b) 

7.4 

28.2 

46.4 

48.9 

44.3 

38.8 

Missing 

2.8 

1.0 

6 1 

1.3 

4.3 

2.9 

Sample size (total = 6.974) 

1.083 

1.071 

1.273 

1.089 

1.424 

1.034 


(continued) 
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Table 1: Baseline Characteristics of Students in Sample, by College (continued) 


SOURCE: \IDRC calculations from Baseline Information Form data. 

NOTES: Calculations for this table used all available data for the 6.974 sample members. 

Random assignment ratios vary across cohorts. Estimates are weighted to account for probabihty' of being assigned to the treatment groi^. 
Charactenstics shown m italics are calculated for a proportion of the full sample. 

Distributions may not add to 100 percent because of rounding. 

Missing values are only included in variable distnbutions for characteristics with more than 3 percent of the sample missing. 

‘Responxlents w'ho said they are Hispamc and chose a race are included only in the Hispamc category. Respondents wiio said they are not Hispamc and 
chose more than one race are only in the multuacial category. 

'’"Other" race, ethmcity includes those who marked "other," more than one race, or .American Nati\,"e Native Alaskan 
TOata on whether sample members received financial aid were not collected at Kmgsborough. 

''Distributions may not add to 100 percent because categories are not mutually exclusive. 

For Kingsborough, only students in the sample who placed into developmental English are included in this table. At the other colleges, the trials 
required students to be enrolled in developmental education for the subject offered in the learning communities. 
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Table 2: Baseline Characteristics of Full Kingsborough Sample 


Characteristic (%) 

Sample 

Gender 

Male 

45.4 

Female 

54.6 

Age 

17-18 years old 

44.5 

19-20 years old 

34.2 

21-34 years old 

21.3 

Marital status 

Married 

3.9 

Unmarried 

96.1 

Race/ethnicity^ 

Hispanic/Latino 

20.4 

Black, non-Hispanic 

37.7 

White, non-Hispanic 

26.9 

Asian or Pacific Islander 

8.6 

Other 

6.4 

Has one or more children 

8.7 

Household receiving any government benefits*’ 

28.4 

Financially dependent on parents 

74.2 

Ever employed 

78.2 

Currently employed 

35.5 

Diplomas/degrees earned*’ 

High school diploma 

70.9 

General Educational Development 

(GED) certificate 

28.6 

Occupational/technical certificate 

2.0 

Date of high school graduation/GED receipt 

During the past year 

70.2 

Between 1 and 5 years ago 

22.8 

More than 5 years ago 

7.0 

Main reason for enrolling in college*’ 

To complete a certificate program 

2.8 

To obtain an associate's degree 

29.7 

To transfer to a 4-year 

college/university 

50.2 

To obtain/update job skills 

10.8 

Other 

8.4 


(continued) 
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Table 2: Baseline Characteristics of Full Kingsborough Sample (continued) 


Characteristic 

Full 

Sample 

First person in family to attend college 

33.4 

Working personal computer in home 

79.7 

Owns or has access to a working car 

25.6 

Language other than English spoken regularly in 

home 

46.9 

U.S. citizen 

72.6 

Respondent born outside U-S."* 

39.9 

Respondent or respondent's parent(s) born outside 

U.S.“ 

74.4 

Region in which respondent was bom 

North America 

60.0 

Asia 

6.3 

Commonwealth of Independent 

States® 

9.5 

Latin America and the Caribbean 

18.7 

Other’ 

5.5 

Region in which respondent's mother was born® 

North America 

28.2 

Asia 

9.8 

Commonwealth of Independent 

States® 

11.0 

Latin America and the Caribbean 

41.5 

Other’ 

9.6 

Sample size (total = 1,534) 

1,534 


SOLTRCE: MDRC calculations usina Baseline InfonnationFonn (BIF) data. 

NOTES: Distributions may not add to 100 percent because ofronnding. Missing values are not 
included in individual variable distributions. 

^Respondentswho indicated that the\' are Hispanic and wiio also chose a race are included only 
in the Hispanic Latino categoiy . 

’’Benefits include nnemployment dislocated worker benefits. Supplemental SecnriU' Income 
(SSI) or disabilit)', cash assistance or welfare, food stamps, and Section 8 or public housing. 
'’Distributions may not add to lOOpercent because categories are not mntnally exclusive. 

'’"U. S . " includes Puerto Rico. 

This commonwealth comprises Amienia, Azerbaijan, Belanis, Georgia (until 2009), 
Kazakhstan, Kyrg\ zstan, Moldova, Russia, Tajikistan, Turkmenistan (unolficial member), flaaine 
(unolficial member), and Uzbekistan. 

^Other regions include the Baltics, Eastern and Western Europe, North Africa, Sub-Saharan 
Africa, the Near East, and Oceania. Countries are giouped by region accordingto the US. Bureau of 
the Census. International Data Base. 

Themajorirt' ofrespondents reported that both parents were bom in the same region as each 

other. 
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Table 3: Key Academic Outcomes at Kingsborough, Six Years after Random Assignment 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Earned a degree" (%) 

35.9 

31.3 

4.6 * 

2.7 

Highest degree earned'’ (%) 

Bachelor's degree 

or higher 

11.8 

9.4 

2.4 

1.8 

Associate's degree 

23.3 

21.1 

2.2 

2.2 

Number of semesters enrolled 

6.1 

5.9 

0.2 

0.2 

Total credits earned" 

56.3 

52.3 

4.0 * 

2.3 

Ever enrolled in a four-year 

institution (%) 

42.4 

42.0 

0.4 

2.7 

Sample size (total = 1,534) 

769 

765 




SOURCE: MDRC calculations from CUNY Institutional Research Database (IRDB) and National Student 
Clearingliouse data. 


NOTES : Roundinaniay cause sliglit discrepancies in sums and dift'erences. 

A two-tailed t-test was applied to differences between research gioups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort. Standard en ors are clustered by leanhng communit}’ link. 

Degi ee and enrollment measures include outcomes from any college. Credits measure refers to credits 
earned at any CUNY college. 

®No one with a known degi ee rt pe earned a certificate, althougli there may have been a small number 
of certificate earners whose degi ee rt pe was unknown. 

’’Percentage who earned bachelor's degj ee or higlier and percentage who earned associate's degj ee do 
not add up to total because degi ee-eainers whose degi ee U pewas unknown were excluded. 

Total credits include both college-level and developmental credits. 
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Table 4: Key Academic Outcomes by English Skills Assessment at Baseline, Six Years after 
Random Assignment 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Difference 

Between 

Subgroups 

Passed both English tests at 
baseline 






Earned a degree^ (%) 

50.3 

37.7 

12.6 *** 

4.4 

t 

Number of semesters 
enrolled 

6.8 

6.4 

0.4 

0.3 


Total credits earned'’ 

64.2 

56.9 

7.3 * 

3.7 


Sample size (total = 445) 

225 

220 




Failed one English test at 
baseline 






Earned a degree® (%) 

32.3 

32.5 

-0.2 

3.2 

t 

Number of semesters 
enrolled 

5.9 

5.8 

0.1 

0.3 


Total credits earned'’ 

53.1 

53.4 

-0.4 

3.1 


Sample size (total = 704) 

347 

357 




Failed both English tests at 
baseline 






Earned a degree® (%) 

26.0 

21.1 

4.9 

5.0 

t 

Number of semesters 
enrolled 

5.6 

5.2 

0.3 

0.4 


Total credits earned'’ 

52.7 

44.7 

8.0 * 

4.3 


Sample size (total = 385) 

197 

188 





SOLTRCE: MDRC calculations from CUNY Institutional Research Database (IRDB) and National Student 
Clearingliouse data. 


NOTES: Roundingmay cause sliglit discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research gioups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences of impacts between subgi oups. Statistical significance 
levels are indicated as: ttt = 1 percent; tt = ^ percent; | = 10 percent. 

Estimates are adjusted by research cohort. Standard enors are clustered by leaming conmninib,’ link. 

Degi ee and enrollment measures include outcomes from any college. Credits measure refers to credits 
earned at any CUNY college. 

®No one with a known degi ee t} pe earned a certificate, althougli there may have been a small number 
of certificate earners whose degi ee t} pe was unknown. 

*>Total credits includeboth college-level and developmental credits. 
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Figure 1: Cumulative Credits Earned in the Targeted Subject by Pooled Sample of 
Developmental Education Students 



■ Proa am Group □ C ontrol Group 


SOURCE: MDRC calculations from the Communin' College of Baltimore County, Hillsborough Commumt>’ 
College. Houston Community College, Kingsborough Communin' College, Merced College, and 
Queensborough Commumty College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and dififeiences. 

A nvo-tailed t-test w'as applied to differences benveen research groups. Statistical significance lei'els are 
indicated as: *** = 1 percent: ** = 5 percent; * = 10 percent. 

The probability of being assigned to the treatment groi^ varies across colleges and within random 
assignment cohorts, and estimates are weighted to account for the diffeient random assignment ratios. Estimates 
are adjusted by campus and cohort. Standard errors are clustered by leammg community link- 
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Figure 2: Cumulative Total Credits Earned by Pooled Sample of Developmental Education 
Students 



■ Program Group □ Control Groi^ 


SOURCE: MDRC calculations from the Communin' College of Baltmiore Cotinty, Hillsborough Commumty 
College. Houston Communin' College, Kingsborough Commumn' College. Merced College, and 
Queensborough Commumty College transcript data 

NOTES: Roundmg may cause shght discrepancies m sums and differences. 

A nvo-tailed t-test %-as apphed to differences benveen research groups. Statistical significance levels are 
mdicated as; *** = 1 percent: ** = 5 percent; * = 10 percent. 

The probabiht>' of being assigned to the treatment group varies across colleges and within random 
assignment cohorts, and estimates are weighted to account for the different random assignment ratios. Estimates 
are adjusted by campus and cohort. Standard errors are clustered by leammg communin' link- 
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Figure 3: Impact of the Learning Communities Program on Credits Earned in the 
Targeted Subject at the End of the Program Semester, by College 








College 


SOURCE: MDRC calculations from the Community College of Baltimore County, Hillsborough Commumty 
College, Houston Commumty College, Kingsborough Community College. Merced College, and Queensborough 
Community College transcript data 
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Figure 4: Impact of the Learning Communities Program on Total Credits Earned at the 
End of the Program Semester, by College 



College 


SOURCE: MDRC calculations from the Community College of Baltimore County. Hillsborough Commumty 
College, Houston Community College, Kmgsborough Commimity College, Merced College, and Queensborough 
Community College transcript data. 
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